Greptile Summary

This PR adds audio attachment support for audio-capable models (Gemini), mirroring the existing image pipeline. Audio is off by default and must be explicitly opted in per-bot via include_audio: true / max_audio, preventing accidental routing to models that cannot handle audio.

• Ingestion (connector.ts): detects audio/* attachments by content_type or filename extension, pre-checks attachment size before downloading (12 MB cap), normalizes MIME to Gemini-accepted types, and caches audio in-memory per session.
• Emission (format-messages.ts): totalBase64Size is promoted to function scope so images and audio share a combined 18 MB inline-data budget, with audio gated by include_audio and bounded by max_audio.
• Adapter + logging (adapter.ts, provider.ts): adds the audio → membrane block conversion and extends the base64-redaction log path to cover audio alongside images.

Confidence Score: 4/5

Safe to merge — audio is off by default and the new paths are well-gated. The only rough edge is the in-memory audio cache having no eviction policy, which is a concern for long-running bots but not an immediate breakage.

The feature is cleanly opt-in, the combined inline-data budget correctly accounts for both images and audio, and base64 redaction in logs was extended to cover the new media type. The audioCache grows without bound over a session's lifetime — each unique audio URL fetched stays in the heap at up to 12 MB raw — but since max_audio defaults to 1 and hits on the same URL are deduplicated, the practical growth rate is low for typical usage. The profiling log also omits the new audioCount field, which is a minor observability gap.

src/discord/connector.ts — audio cache lifetime and the profiling log omission are both here.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant Discord
    participant Connector as connector.ts
    participant AudioCache as audioCache (in-memory)
    participant Builder as context/builder.ts
    participant Format as format-messages.ts
    participant Adapter as membrane/adapter.ts
    participant Gemini

    Discord->>Connector: Message with audio attachment
    Connector->>Connector: isAudioAttachment() — content_type or extension
    Connector->>Connector: "Pre-fetch size check (attachment.size > 12 MB → skip)"
    Connector->>Connector: audioMimeFor() — normalize MIME
    Connector->>AudioCache: cacheAudio(url, mediaType, messageId, duration)
    AudioCache->>Connector: CachedAudio (Buffer + metadata)
    Connector->>Builder: "DiscordContext { audios: CachedAudio[] }"
    Builder->>Format: formatMessages(..., audios)
    Format->>Format: Build audioMap (url → CachedAudio)
    Format->>Format: "Check include_audio + audioEmitted < maxAudio"
    Format->>Format: Check combined inline budget (images + audio ≤ 18 MB)
    Format->>Adapter: AudioContent block (base64 + media_type)
    Adapter->>Gemini: "{ type: audio, source: { type: base64, mediaType, data } }"

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant Discord
    participant Connector as connector.ts
    participant AudioCache as audioCache (in-memory)
    participant Builder as context/builder.ts
    participant Format as format-messages.ts
    participant Adapter as membrane/adapter.ts
    participant Gemini

    Discord->>Connector: Message with audio attachment
    Connector->>Connector: isAudioAttachment() — content_type or extension
    Connector->>Connector: "Pre-fetch size check (attachment.size > 12 MB → skip)"
    Connector->>Connector: audioMimeFor() — normalize MIME
    Connector->>AudioCache: cacheAudio(url, mediaType, messageId, duration)
    AudioCache->>Connector: CachedAudio (Buffer + metadata)
    Connector->>Builder: "DiscordContext { audios: CachedAudio[] }"
    Builder->>Format: formatMessages(..., audios)
    Format->>Format: Build audioMap (url → CachedAudio)
    Format->>Format: "Check include_audio + audioEmitted < maxAudio"
    Format->>Format: Check combined inline budget (images + audio ≤ 18 MB)
    Format->>Adapter: AudioContent block (base64 + media_type)
    Adapter->>Gemini: "{ type: audio, source: { type: base64, mediaType, data } }"

Fix the following 2 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 2
src/discord/connector.ts:111
**Unbounded in-memory audio cache**

`audioCache` is a `Map` that grows without eviction for the life of the connector instance. Unlike `imageCache`, which has a disk persistence layer and URL map as a backing store, audio lives purely in RAM. Each entry can be up to 12 MB; a busy bot processing many distinct audio URLs across a long session can accumulate significant heap pressure. The image cache has the same pattern, but images are typically re-encountered across turns (cache hits) whereas audio files sent once are rarely re-sent, meaning the hit rate for audio may be lower and the unique-URL accumulation rate higher.

### Issue 2 of 2
src/discord/connector.ts:907-914
**Missing `audioCount` in the profiling and attachment-completion logs**

The `⏱️ PROFILING` log records `imageCount` and `documentCount` but not `audioCount`. When audio is silently dropped or not fetched, this makes the log entry misleading — a bot with `include_audio: true` shows no sign in the profiling line of whether audio was actually collected.

```suggestion
      // Log fetch timings
        logger.info({
        ...timings,
        messageCount: discordMessages.length,
        imageCount: images.length,
        documentCount: documents.length,
        audioCount: audios.length,
        pinnedCount: pinnedConfigs.length,
      }, '⏱️  PROFILING: fetchContext breakdown (ms)')
```

_{Reviews (1): Last reviewed commit: "fix(audio): emit extension-detected audi..." | Re-trigger Greptile}

Thanks Greptile — both addressed in 5053bb8:

1. Unbounded audio cache → audioCache now has an LRU byte budget (AUDIO_CACHE_MAX_BYTES, 64 MB): cache hits refresh recency, and evictAudioCache() drops oldest entries once over budget. This keeps the cross-activation dedup benefit for audio still in the rolling context window while bounding heap. (Note: imageCache shares the original unbounded pattern; left as-is to keep this PR scoped, but it's a reasonable follow-up.)
2. Missing audioCount → added to both the ⏱️ PROFILING and "Attachment processing complete" logs.

Good catches — the audio cache distinction (lower hit rate, no disk backing, larger entries) was a fair call.

✅ Acceptance-tested (2026-07-04)

Verified live on a Discord server, together with antra-tess/membrane#29:

• Gemini 3.5 Flash (gemini-3.5-flash, native Gemini vendor, inlineData path): described an attached mp3 in detail — genre, instrumentation, vocals, lyric themes, per-timestamp structure.
• MiMo V2.5 (xiaomi/mimo-v2.5 via OpenRouter, input_audio path, provider-pinned to Parasail/Venice/DeepInfra): equally detailed track breakdown.

Found during testing and fixed in 31915fd: Discord reports mp3 attachments as the legacy MIME alias audio/mpeg3 in the wild — now normalized to audio/mp3.

Gemini 3.5 Flash:

MiMo V2.5 (OpenRouter):

🤖 Generated with Claude Code

Follow-up: size-cap UX + payload sizing docs (fd447a6)

• Oversized audio attachments (> 12 MB per-file cap) were skipped with only a server-side log line, so users got a misleading text-only reply. The bot now reacts to the message with a configurable emote (default 🐘, oversized_audio_emote: '' disables), mirroring the existing 🚫 blocked-message reaction; deduped per message so rolling-window rescans don't re-react.
• claude.yaml.example now documents payload sizing: the per-file cap × max_audio all ships base64-inlined in one request, so max_audio: 1 is the recommended default while the per-file cap sits at ~12–15 MB; raising max_audio means dividing the ~15 MB raw-audio budget across files.

🤖 Generated with Claude Code